BY-COVID - WP5 - Baseline Use Case: COVID-19 vaccine effectiveness assessment

Survival analysis

Survival plot

We estimate the survival function using the Kaplan-Meier estimator and represent this function visually using a Kaplan-Meier curve, showing the probability of not getting infected by SARS-CoV-2 at a certain time after onset of follow-up. The survival function is estimated for the control and intervention group.


The cumulative incidence of the event (SARS-CoV-2 infection) was additionally plotted.


Survival (time-to-event)

The probability of not getting infected by SARS-CoV-2 beyond a certain time after onset of follow-up (survival function, estimated using the Kaplan-Meier estimator) is reported for different periods.

Strata Time Number at risk Cumulative sum of number of events Cumulative sum of number censored Survival Std. error Cumulative hazard Std. error cumulative hazard
Not fully vaccinated 0 7757 0 0 1.0000 0.00000 0.0000 0.00000
Not fully vaccinated 100 2629 55 5092 0.9869 0.00186 0.0132 0.00188
Not fully vaccinated 200 861 193 6721 0.9053 0.00716 0.0995 0.00790
Not fully vaccinated 300 595 301 6861 0.7684 0.01360 0.2632 0.01767
Not fully vaccinated 400 461 324 6984 0.7383 0.01444 0.3030 0.01953
Not fully vaccinated 500 84 325 7350 0.7364 0.01453 0.3056 0.01970
Fully vaccinated 0 7757 0 0 1.0000 0.00000 0.0000 0.00000
Fully vaccinated 100 2628 60 5086 0.9857 0.00193 0.0144 0.00196
Fully vaccinated 200 841 249 6684 0.8737 0.00822 0.1349 0.00939
Fully vaccinated 300 603 334 6820 0.7676 0.01300 0.2642 0.01691
Fully vaccinated 400 482 352 6931 0.7447 0.01369 0.2945 0.01836
Fully vaccinated 500 89 352 7318 0.7447 0.01369 0.2945 0.01836


Median survival time

The median survival time is the time corresponding to a probability of not obtaining a SARS-CoV-2 infection probability of 0.5. (if NA, the probability of not obtaining a SARS-CoV-2 infection did not drop below 50%)

Characteristic Median survival (95% CI)
fully_vaccinated_bl
    FALSE — (—, —)
    TRUE — (—, —)
Cox regression and estimation of the average treatment effect

A Cox regression model was built to examine the relationship between the distribution of the probability of not obtaining a SARS-CoV-2 infection (survival distribution) and completing a primary vaccination schedule (covariate). The Cox proportional hazards regression model was fitted with ‘fully_vaccinated_bl’ as a covariate and accounts for clustering within individuals (as one individual can be re-sampled as control).

A hazard ratio (HR) is computed for the covariate ‘fully_vaccinated_bl’. A hazard can be interpreted as the instantaneous rate of SARS-CoV-2 infections in individuals that are at risk for obtaining an infection (Cox proportional hazards regression assumes stable proportional hazards over time). A HR < 1 indicates reduced hazard of SARS-CoV-2 infection when having completed a primary vaccination schedule whereas a HR > 1 indicates an increased hazard of SARS-CoV-2 infection.

Parameter estimate SE coefficient Robust SE coefficient P-value Hazard Ratio (HR) (95% CI for HR)
fully_vaccinated_blTRUE 0.083 0.077 0.1 0.41 1.086 (0.873, 1.3)

The overall significance of the model is tested.

Test statistic Df P-value
Likelihood ratio test 1.1538840 1 0.2827376
Wald test 0.6800000 1 0.4103826
Score (logrank) test 1.1536006 1 0.2827967
Robust score test 0.6978656 1 0.4035018

Proportional hazards during the study period might be unlikely. As such, the RMST and RMTL ratios are additionally calculated, providing an alternative estimate for the the Average Treatment Effect (ATE), without requiring the proportional hazards assumption to be met.

Arm Measure Estimate SE CI.lower CI.upper
fully_vaccinated_bl==FALSE RMST 326.369 2.049 322.354 330.384
fully_vaccinated_bl==TRUE RMST 324.198 2.045 320.189 328.207
fully_vaccinated_bl==FALSE RMTL 38.631 2.049 34.616 42.646
fully_vaccinated_bl==TRUE RMTL 40.802 2.045 36.793 44.811
Measure Estimate CI.lower CI.upper p_value
RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -2.171 -7.845 3.503 0.453
RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.993 0.976 1.011 0.453
RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.056 0.915 1.219 0.454
Survival plot

We estimate the survival function using the Kaplan-Meier estimator and represent this function visually using a Kaplan-Meier curve, showing the probability of not getting infected by SARS-CoV-2 at a certain time after onset of follow-up. The survival function is estimated for the control and intervention group, within subsets defined by the vaccination schedule.


The cumulative incidence of the event (SARS-CoV-2 infection) was additionally plotted within subsets defined by the vaccination schedule.


Cox regression and estimation of the average treatment effect

A Cox regression model was built to examine the relationship between the distribution of the probability of not obtaining a SARS-CoV-2 infection (survival distribution) and completing a primary vaccination schedule (covariate), and whether this differs according to the administered vaccination schedule. A stratified Cox proportional hazards regression model was fitted with ‘fully_vaccinated_bl’ as a covariate, ‘vaccination_schedule_cd’ as a stratification factor, and accounting for clustering within individuals (as one individual can be re-sampled as control).

Parameter estimate SE coefficient Robust SE coefficient P-value Hazard Ratio (HR) (95% CI for HR)
fully_vaccinated_blTRUE 0.012 0.212 0.220 0.956 1.012 (0.576, 1.449)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)BP-BP 0.182 0.232 0.226 0.420 1.2 (0.668, 1.732)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)JJ -0.236 0.425 0.421 0.576 0.79 (0.137, 1.442)
fully_vaccinated_blTRUE:strata(vaccination_schedule_cd)MD-MD -0.241 0.285 0.280 0.390 0.786 (0.354, 1.218)


The overall significance of the model is tested.

Test statistic Df P-value
Likelihood ratio test 6.037588 4 0.1963587
Wald test 5.620000 4 0.2291565
Score (logrank) test 6.029537 4 0.1969533
Robust score test 5.777951 4 0.2163561

Proportional hazards during the study period might be unlikely. As such, the RMST and RMTL ratios are additionally calculated, providing an alternative estimate for the the Average Treatment Effect (ATE), without requiring the proportional hazards assumption to be met.

Vaccination_schedule Arm Measure Estimate SE CI.lower CI.upper
BP-BP fully_vaccinated_bl==FALSE RMST 328.928 2.429 324.166 333.690
BP-BP fully_vaccinated_bl==TRUE RMST 324.322 2.463 319.495 329.149
BP-BP fully_vaccinated_bl==FALSE RMTL 36.072 2.429 31.310 40.834
BP-BP fully_vaccinated_bl==TRUE RMTL 40.678 2.463 35.851 45.505
MD-MD fully_vaccinated_bl==FALSE RMST 316.357 5.728 305.131 327.584
MD-MD fully_vaccinated_bl==TRUE RMST 325.702 5.217 315.477 335.927
MD-MD fully_vaccinated_bl==FALSE RMTL 48.643 5.728 37.416 59.869
MD-MD fully_vaccinated_bl==TRUE RMTL 39.298 5.217 29.073 49.523
AZ-AZ fully_vaccinated_bl==FALSE RMST 322.389 6.010 310.609 334.169
AZ-AZ fully_vaccinated_bl==TRUE RMST 319.210 6.266 306.930 331.490
AZ-AZ fully_vaccinated_bl==FALSE RMTL 42.611 6.010 30.831 54.391
AZ-AZ fully_vaccinated_bl==TRUE RMTL 45.790 6.266 33.510 58.070
JJ fully_vaccinated_bl==FALSE RMST 330.197 8.954 312.647 347.746
JJ fully_vaccinated_bl==TRUE RMST 330.886 9.081 313.087 348.685
JJ fully_vaccinated_bl==FALSE RMTL 34.803 8.954 17.254 52.353
JJ fully_vaccinated_bl==TRUE RMTL 34.114 9.081 16.315 51.913
Vaccination_schedule Measure Estimate CI.lower CI.upper p_value
BP-BP RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -4.606 -11.386 2.175 0.183
BP-BP RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.986 0.966 1.007 0.183
BP-BP RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.128 0.944 1.347 0.185
MD-MD RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 9.345 -5.840 24.530 0.228
MD-MD RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.030 0.982 1.079 0.228
MD-MD RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.808 0.571 1.144 0.229
AZ-AZ RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) -3.179 -20.196 13.838 0.714
AZ-AZ RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.990 0.939 1.044 0.714
AZ-AZ RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.075 0.731 1.580 0.714
JJ RMST (fully_vaccinated_bl==TRUE)-(fully_vaccinated_bl==FALSE) 0.690 -24.306 25.685 0.957
JJ RMST (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 1.002 0.929 1.081 0.957
JJ RMTL (fully_vaccinated_bl==TRUE)/(fully_vaccinated_bl==FALSE) 0.980 0.474 2.025 0.957